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Executive Summary 



Nationally, relatively few of the more than 2.5 million adults who enroll annually 
in basic skills programs advance successfully to college-level coursework. This limits the 
ability of such individuals to secure jobs that pay family-supporting wages and that offer 
opportunity for career advancement. This paper presents findings from a study conducted 
by the Community College Research Center (CCRC) at Teachers College, Columbia 
University, on the outcomes of the Integrated Basic Education and Skills Training 
program, or I-BEST, an innovative program developed by the community and technical 
colleges in Washington State to increase the rate at which adult basic skills students enter 
and succeed in postsecondary occupational education and training. 

Under the I-BEST model, basic skills instructors and college-level career- 
technical faculty jointly design and teach college-level occupational courses for adult 
basic skills students. Instruction in basic skills is thereby integrated with instruction in 
college-level career-technical skills. The I-BEST model challenges the conventional 
notion that basic skills instruction ought to be completed by students prior to starting 
college-level courses. The approach thus offers the potential to accelerate the transition of 
adult basic skills students to college programs. 

The CCRC study reported on here used multivariate analysis to compare the 
educational outcomes over a two-year tracking period of I-BEST students with those of 
other basic skills students, including students who comprise a particularly apt comparison 
group — those non-I-BEST basic skills students who nonetheless enrolled in at least one 
workforce course in academic year 2006-07, the period of enrollment in the study. The 
researchers examined data on more than 31,000 basic skills students in Washington State, 
including nearly 900 I-BEST participants. The analyses controlled for observed 
differences in background characteristics of students in the sample. 

The study found that students participating in I-BEST achieved better educational 
outcomes than did other basic skills students, including those who enrolled in at least one 
non-I-BEST workforce course. I-BEST students were more likely than others to: 

• Continue into credit-bearing coursework; 

• Earn credits that count toward a college credential; 
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• Earn occupational certificates; and 

• Make point gains on basic skills tests. 

On all the outcomes examined, I-BEST students did moderately or substantially better 
than non-I-BEST basic skills students in general. The I-BEST group’s comparative 
advantage relative to non-I-BEST basic skills students who enrolled in at least one 
workforce course was not as large, but was still significant. 

The study also compared I-BEST students to a group of non-participants with 
similar characteristics who were matched with the I-BEST students using a statistical 
technique called propensity score matching (PSM). Using the PSM analysis, the study 
estimated that, over the two-year tracking period, the probability that I-BEST students 
would earn at least one college credit was 90 percent, while the probability for the 
matched students was 67 percent, a 23 percentage point difference. I-BEST students 
earned, on average, an estimated 52 quarter-term college credits, compared to an average 
of 34 quarter-term credits for the matched comparison group. I-BEST students had a 
higher probability of persisting into the second year: 78 percent, compared with 61 
percent for the matched group. The chances of earning an occupational certificate was 55 
percent for I-BEST students, compared with only 15 percent for the matched group. T 
BEST students also had a higher likelihood of making point gains on the CASAS basic 
skills test: 62 percent compared with 45 percent for the matched group. 

While the results of this analysis show that participation in I-BEST is correlated 
with better educational outcomes over the two-year tracking period, it is important to note 
that they do not provide definitive evidence that the I-BEST program caused the superior 
outcomes. It could be that, because of the way students are selected into the program, 
those who participate have higher motivation or other characteristics not measured in this 
study that make them more likely to succeed. Selection bias could also operate in the 
other direction if I-BEST students are more disadvantaged in ways we do not measure. 

In the future, CCRC researchers plan to conduct fieldwork to better understand 
the process by which students are selected into the program. CCRC will also extend this 
study by examining degree attainment and labor force outcomes of I-BEST students over 
a longer timeframe, by collecting financial data to estimate program cost-effectiveness, 
and by examining the practices of I-BEST programs that produce superior outcomes. 
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1. Introduction and Background 



Today, most jobs that pay wages sufficient to support a family require at least 
some postsecondary education, and, increasingly, a college credential (Barton, 2000; 
Baum & Ma, 2007; Osterman, 2008). Yet, according to the Census Bureau (2007a), more 
than 75 million Americans 25 years and older have no education beyond high school. 
More than 30 million of them lack a high school credential or GED. In addition, over 12 
million individuals in this age group lack basic fluency in English (U.S. Census Bureau, 
2007b), which also limits the ability to secure a good job. 

Community colleges, schools, and community organizations offer programs for 
adults with limited skills and education, including adult basic education (ABE) and GED 
preparation programs for individuals who do not have a high school credential, and 
English-as-a-second-language (ESE) programs for those with limited proficiency in 
English. Over 2.5 million students enroll each year in adult basic skills programs funded 
by states and the federal government (U.S. Department of Education, 2006). Yet, few 
students in these programs advance to college-level education and training, even if they 
are enrolled in a program offered at a community college. 

Eor example, a study conducted in Washington State (Prince & Jenkins, 2005), 
where ABE and ESE programs are delivered through community and technical colleges, 
found that only 3 1 percent of a cohort of students who started in ABE earned at least one 
college credit in five years. The comparable rate for students who started in ESE was 
only 12 percent. This same study found that, compared with students who earned fewer 
than 10 college credits, those who reached the “tipping point” of earning at least two 
semesters of credits and a credential had a substantial average annual earnings advantage: 
$7,000 for students who started in ESE, $8,500 for those who started in ABE or GED, 
and $2,700 and $1,700 for those who entered with at most a GED or high school 
diploma, respectively. 

One reason that few students in adult basic skills programs advance successfully 
to college-level coursework is that such programs are typically not well aligned with 
college-level offerings. Adult basic skills students often do not have access to counselors 
and other supports available to students in college programs. Moreover, many students in 
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adult basic skills programs face the challenge of having to balance school and family 
while working one or more low-wage jobs. 

The study reported on here, conducted by the Community College Research 
Center (CCRC) at Teachers College, Columbia University, examined the outcomes of an 
innovative approach to increasing the rate at which adult basic skills students enter and 
succeed in postsecondary education and training. The model, known as Integrated Basic 
Education and Skills Training, or I-BEST, was developed by the Washington State 
community and technical colleges, based on the conviction that helping the state’s 
growing number of poorly educated immigrants and other low-skill adults succeed in 
postsecondary training connected to jobs of importance to local economies had become 
an economic imperative (Bloomer, 2008; WSBCTC, 2005). In the I-BEST model, basic 
skills instructors and college-level career-technical faculty jointly design and teach 
college-level occupational (or what the Washington community and technical colleges 
call “workforce”) courses for adult basic skills students. ^ Under I-BEST, instruction in 
basic skills is combined with instruction in college-level career- technical skills. Students 
receive college credit for the workforce portion of the program (though not for the basic 
skills instruction). 

The program design was motivated by research suggesting that teaching basic 
skills in the context of materials that are of interest to the student — sometimes called 
“contextual instruction” — can improve learning of basic skills by adults (Resnick, 1987; 
Shore, Shore, & Boggs, 2004; Sticht, 1997; Stone, Alfred, Pearson, Eewis, & Jensen, 
2005; Weinbaum & Rogers, 1995). In I-BEST, basic skills instruction is typically 
customized to the given workforce program. Eor example, for students enrolled in a 
nursing program, there may be increased emphasis on learning medical terms in addition 
to mastering everyday vocabulary used in all fields. If a student is having difficulty 
understanding technical material because of problems with English, the basic .skills 
instructor is there to help. The theory is that student motivation and achievement will 
increase because students are able to immediately experience the usefulness of their basic 
skills education in the learning of technical skills and knowledge. 



* Both instructors of a course are required to be present in class for at least half of the total instructional 
time. 
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Integrating the teaching of basic skills and college-level career-technical skills, as 
is done through the I-BEST model, challenges the conventional notion that basic skills 
instruction should be completed prior to starting college-level courses. The approach thus 
offers the potential to accelerate the rate at which basic skills students advance to college 
programs. Preliminary analyses of TBEST program outcomes by researchers at the 
Washington State Board for Community and Technical Colleges (WSBCTC) found that 
participating students were substantially more likely than non-participating adult basic 
skills students to advance to college-level workforce programs and to reach the “tipping 
point” of having earned at least one year of credits and a credential (WSBCTC, 2005, 
2008). 

Based on these promising early results, the WSBCTC approved increased funding 
of programs using the I-BEST model. I-BEST courses receive 75 percent more funds per 
full-time-equivalent student than do regular basic skills courses. With this enhanced 
funding, the program model has expanded from pilots at 5 colleges in 2004-05 to 
programs at all 34 community and technical colleges in the Washington State system. 
Over 115 I-BEST programs are currently offered in such fields as nurse assistant, early 
childhood education, and business technology (Bloomer, 2008). Table 1 shows the fields 
in which I-BEST students enrolled in 2006-07. The WSBCTC requires that credits earned 
in I-BEST programs, which are typically a single quarter term in length, should apply to 
certificate or degree programs that are part of a “career pathway,” that is, programs that 
clearly connect to further education and career-path employment in the given field. 



^ The Washington community and technical colleges operate on a quarter system. 
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Table 1. 1-BEST Enrollments by Program Description, 2006-07^ 

Program Description Enrollment 

Data Entry/Microcomputer Applications (General) 172 

Early Childhood Education and Teaching 92 

Nurse/Nursing Assistant/ Aide and Patient Care Assistant 74 

Automobile/ Automotive Mechanics Technology/Technician 73 

Welding Technology/W elder 7 1 

Medical/Clinical Assistant 7 1 

Criminal Justice/Law Enforcement Administration 54 

Home Health Aide/Home Attendant 42 

Nursing/Registered Nurse (RN, ASN, BSN, MSN) 20 

Medical Office Management/ Administration 18 

Truck and Bus Driver/Commercial Vehicle Operation 17 

Medical Reception/Receptionist 17 

Occupational Safety and Health Technology/Technician 15 

Office Management and Supervision 15 

Accounting Technology/Technician and Bookkeeping 13 

Business/Office Automation/Technology/Data Entry 12 

Electrical, Electronic and Communications Engineering ^ ^ 

Technology/Technician 

Medical Administrative/Executive Assistant and Medical ^ 

Secretary 

Graphic and Printing Equipment Operator, General ^ 

Production 

Manufacturing Technology/Technician 4 

Natural Resources Law Enforcement and Protective Services 4 

Eorensic Science and Technology 2 

Executive Assistant/Executive Secretary 2 

Business Administration and Management, General 1 



^ Due to limitations in the administrative data, 87 of the 896 I-BEST students enrolled in 2006-07 were 
lacking information on which vocational program they were enrolled in; this table thus reflects the program 
enrollments of 809 I-BEST participants. 
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The early analyses of I-BEST programs by the WSBCTC were descriptive in 
nature, in that they did not control for student characteristics that could bear on outcomes. 
In addition, those analyses did not consider that the way students are selected into 
programs may influence the results. According to program administrators, I-BEST 
students most often find out about I-BEST through word of mouth or by participating in a 
non-I-BEST basic skills course (either ABE or ESE). Students are also referred by 
persons affiliated with a given college, such as counselors, as well as by outside entities. 
Organizations that have referred students to I-BEST include WorkSource, which is 
Washington State’s system of public “one-stop” employment centers, WorkEirst, 
Washington State’s program for connecting public aid recipients to jobs, and various 
retraining programs for dislocated workers. Einally, students may also apply directly to T 
BEST programs, which are publicized through such means as college websites, catalogs, 
and flyers. 

Students are therefore selected into I-BEST in a non-random manner. WSBCTC 
staff members have indicated that the program may be better suited to individuals with 
higher basic skills proficiencies. The program may also attract students who are more 
motivated than others with similar backgrounds and preparation for success in their 
education or career. To accurately estimate the effect of I-BEST on student outcomes, 
what is ideally needed is a comparison group of non-I-BEST students who are similar to 
the students who enroll in I-BEST in all relevant dimensions other than their enrollment 
in the program. 

Eor this study, we chose as the main comparison group those basic skills students 
who, on their own accord, also took at least one (non-I-BEST) college-level career- 
technical course. We refer to this group as Non-IB Workforce students. This is a small 
subset of all basic skills students in the Washington State community and technical 
college system. Of the comparison groups that we have ready access to in terms of 
available data, this is the one most closely comparable to the I-BEST students because 
they, like the I-BEST students, were exposed to both basic skills instruction and 
workforce classes during the period under study. We know, however, that student 
selection into this comparison group operated differently than in the I-BEST group, most 
obviously because the two groups differ on observed characteristics, and, secondly. 
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because they likely differ on characteristics that we do not observe. It is important to keep 
in mind these caveats regarding sample selection when considering our results. 

This study used multivariate analysis to compare the educational outcomes of I- 
BEST students with those of other students with similar characteristics. Our analysis 
addressed two main questions: 

1) What are the socioeconomic, demographic, and enrollment 
characteristics of I-BEST students compared with other basic 
skills students? 

2) How do the educational outcomes of I-BEST students compare 
with those of other basic skills students and, in particular, with 
those in the group mentioned above — students who take at least 
one workforce course but who are not involved in I-BEST? 
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2. Data and Methods 



The data used in this study were drawn from administrative data shared with the 
Community College Research Center (CCRC) by the WSBCTC on both I-BEST and 
non-I-BEST students who enrolled at any college in Washington State’s community and 
technical college system at any time during the academic year 2006-07. We chose to 
study students who enrolled in 2006-07 because WSBCTC staff indicated that this was 
the first year that the program moved beyond the pilot phase and was in full operation. 

We restricted our study to those students who took a non-credit adult basic skills course 
(including, of course, the I-BEST students themselves) in that academic year. We did not 
include the many students who were enrolled in programs designed to prepare for transfer 
to baccalaureate programs because I-BEST programs exist only in occupational fields. 

We also restricted our study to students in the 24 colleges that offered I-BEST in 2006-07 
(the program was expanded to all 34 colleges the following year). We studied data on 
31,078 students, of whom 896 were I-BEST students. Of the remainder, 28,826 were 
students who enrolled only in basic skills courses, and 1,356 were basic skills students 
who also enrolled in at least one (non-I-BEST) workforce course. 

The dataset contains information on the socioeconomic and demographic 
characteristics of each student in the sample, as well as transcript data, which we used to 
determine the number of credits completed and credentials earned. The transcript data 
enabled us to track students through the end of the academic year 2007-08 and back to 
the earliest date each student enrolled in the system, making it possible to control for any 
credits earned prior to 2006-07. 

The study was designed to examine the effects of participation in I-BEST on the 
following educational outcomes over two years: 

• Whether a student earned any college credits; 

• The total number of college credits earned; 

• The number of college vocational credits earned; 

• Whether the student persisted into the following academic year; 

• Whether the student earned a certificate or associate degree; and 

• Whether the student achieved gains on basic skills tests. 
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For each of these outcomes, we produced descriptive statistics comparing I-BEST 
students with two groups: 1) all basic skills students not in I-BEST (“Non-I-BEST 
students”) and 2) those basic skills students not in I-BEST who took at least one 
workforce course during 2006-07 (“Non-IB Workforce students”). Note that we also 
make reference to basic skills students not in I-BEST who did not enroll in a workforce 
course (“Non-IB Non-Workforce students”). 

We then performed regressions to compare: a) I-BEST students and Non-IB Non- 
Workforce students, b) Non-IB Workforce students and Non-IB Non-Workforce 
students, and c) I-BEST students and Non-IB Workforce students. We used linear 
regression or logistic regression, depending on the outcome. In each case, we controlled 
for student characteristics and enrollment patterns that might bear on the outcomes. Eor 
the logistic outcomes, we measured differences in the probability of the given outcome 
between each pair of groups. Eor the continuous outcomes, we measured the differences 
in the outcome itself. 

Einally, we compared the outcomes of I-BEST students with those of Non-I- 
BEST basic skills students who were matched to the I-BEST students using propensity 
score matching (PSM). See the appendix for a brief description of this method. We used 
both regression analysis and PSM to see how similar the results from the two methods 
would be and thus make as an informal test of the robustness of our findings, although 
the two methods cannot be directly compared and draw on different groups of students. 
Eor reasons described in the appendix, we give more credence to the estimates of 
treatment effects produced by PSM than to the results of the regressions. Neither 
regression analysis nor PSM allows us to correct for selection bias that might be caused 
by characteristics we do not observe or measure, however. This remains a limitation of 
this study. 

We consider the treatment in this study to be enrollment in I-BEST, rather than 
completion of an I-BEST program, because we want to view any program attrition effects 
as part of the program itself; that is, we want in our estimates of program effects to 
account for how successful I-BEST was at retaining students. Nevertheless, we have been 
informed by WSBCTC staff that I-BEST programs have high retention and completion 
rates. 
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In all of the tables presented here, we report only the main effects and omit the 
effects for the controls. We used the same set of controls for all of the regressions and in 
our propensity score models. The means for all of the control variables are listed in Table 
4, which is discussed in the next section. 



3. Findings 

We start by giving descriptive statistics on the I-BEST students, the Non-IB Non- 
Workforce students, and the Non-IB Workforce students in our sample. We then present 
results of the multivariate analyses for each outcome. 

3.1 Descriptive characteristics and outcomes 

Overall, 896 I-BEST students were enrolled at 24 community or technical 
colleges in Washington State in academic year 2006-07 (see Table 2). In this study, all 
Non-I-BEST students as well as all those who did enroll in I-BEST programs were, by 
definition, enrolled in basic .skills coursework. Of the 30,182 Non-I-BEST students in the 
sample, 1,356 also took a workforce course. Thus, like the I-BEST students, the latter 
enrolled in both basic skills and workforce coursework in 2006-07. However, unlike the 
I-BEST students, they did not necessarily take the coursework concurrently, and they did 
not take it as part of an integrated program designed to accelerate the transition from 
basic skills to college-level workforce programs. These Non-IB Workforce students 
comprise the group that we believe is most comparable to the I-BEST group. 



Table 2. Distribution of Basic Skills Students, 2006-07 



Program Type 


Enrollment 


I-BEST 


896 


Non-IB Non-Workforce 


28,826 


Non-IB Workforce 


1,356 


Total 


31,078 
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Table 3 shows that I-BEST students were much more likely than Non-I-BEST 
students to advance to college-level coursework and to earn many more college and 
vocational credits. This result is descriptive in nature and does not control for differences 
among students in these groups. I-BEST students were also more likely than Non-IB 
Workforce students to advance and to earn more college and vocational credits. Again, 
this result is descriptive. 

As Table 3 indicates, over the course of the two-year observation period, I-BEST 
students completed slightly more than an academic year’s worth of college coursework, 
on average, while Non-I-BEST students earned very few credits. Of these students, the 
Non-IB Workforce subset accumulated many more credits, on average, than the rest of 
the Non-I-BEST students, but not as many as the I-BEST students. Table 3 also shows 
that 54 percent of I-BEST students earned a certificate or degree, as opposed to less than 
one percent of all Non-I-BEST students and 18 percent of Non-IB Workforce students. 
Virtually all awards earned by anyone in these groups were occupational certificates, not 
degrees. 



Table 3. Educational Outcomes of Basic Skills Students over Two Academic Years, 2006-07 
and 2007-08 



Student type 


Earned 

any 

college 

credit 


Mean 

number 

of 

college 

credits 


Earned 

any 

vocational 

credit 


Mean 
number of 
vocational 
credits 


Earned a 
certificate 


Earned 

an 

associate 

degree 


Earned a 
certificate 
or 

associate 

degree 


I-BEST 


90.0% 


48.7 


87.8% 


41.5 


54.1% 


0.2% 


54.2% 


All Non-I-BEST 


7.0% 


2.3 


5.3% 


1.4 


0.8% 


0.0% 


0.8% 


Non-IB Workforce 


64.2% 


35.7 


59.6% 


24.8 


17.8% 


0.2% 


18.0% 



Table 4 lists the background characteristics that were used as control variables in 
the multivariate models. In some ways, I-BEST students appear to be relatively similar to 
Non-I-BEST students. There are, however, differences worth noting. All basic skills 
students, whether or not they enrolled in I-BEST, were enrolled in either ESE or ABE, 
with the latter possibly including a GED component. Table 4 shows that ABE/GED 
enrollment was dominant among I-BEST students. More than two thirds of I-BEST 
students were enrolled in ABE (or GED) instruction, compared to 36 percent of Non-I- 
BEST students. These proportions are reversed when comparing students in the groups 
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who enrolled in ESL (31 percent of I-BEST versus 64 percent of Non-I-BEST students, 
respectively, were enrolled in ESE). Other differences of note are the percentage of 
students receiving financial aid and the percentage enrolled full time. In both cases, I- 
BEST students held an advantage in that they were more likely to receive aid and enroll 
full time. In terms of race/ethnicity, I-BEST students were much less likely than Non-I- 
BEST students to be Hispanic and much more likely to be Black. 

There are also noteworthy similarities and differences between I-BEST students 
and the Non-IB Workforce student subset. Both the I-BEST and the Non-IB Workforce 
students were mainly ABE/GED students, as opposed to the Non-I-BEST students as a 
whole, who were predominantly ESE students. Non-IB Workforce students were also 
more likely than others to indicate upon entry that they intended to earn an academic 
credential or transfer to a four-year institution. Twenty percent of Non-IB Workforce 
students indicated this, compared to 7 percent of I-BEST students and 9 percent of all 
Non-I-BEST students. 
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Table 4. Characteristics of Basic Skills Students, 2006-07 







All 


Non-IB 




I-BEST 


Non-I-BEST 


Workforce 


Number of students in program 


896 


30,182 


1,356 


Program classification 
I-BEST student 


100% 


0.0% 


0.0% 


ABE/GED student 


69.0% 


36.0% 


66.4% 


ESL student 


30.9% 


63.8% 


33.3% 


Non-IB Workforce student 


0.0% 


4.5% 


100.0% 


Social and economic characteristics 






Mean age 


32.5 


32.3 


31.9 


Eemale 


64.8% 


60.5% 


69.2% 


Hispanic 


18.4% 


38.3% 


21.3% 


Black, non-Hispanic 


12.1% 


6.9% 


6.1% 


Asian/Pacific Islander 


12.3% 


15.0% 


12.4% 


Single w/ dependent 


22.2% 


14.0% 


22.8% 


Married w/ dependent 


27.8% 


26.5% 


24.1% 


Disabled 


7.1% 


3.8% 


11.0% 


Estimated socioeconomic 
status"* 


3.6 


3.5 


3.5 


Current schooling characteristics 
Intent is vocational^ 


72.4% 


22.7% 


48.4% 


Intent is academic 


7.4% 


9.1% 


20.0% 


Received aid 


25.9% 


2.1% 


14.2% 


Enrolled full time 


67.1% 


32.6% 


49.0% 


Eirst enrolled in 1 st quarter 


30.1% 


27.5% 


40.0% 


Eirst enrolled in 2nd quarter 


41.0% 


33.1% 


40.2% 


Eirst enrolled in 3rd quarter 


18.5% 


22.5% 


15.6% 


Eirst enrolled in 4th quarter 


10.4% 


16.9% 


4.2% 


Previous schooling characteristics 
Mean college credits 


13.9 


0.9 


00 

00 


Mean vocational credits 


9.1 


0.6 


5.8 


GED 


12.7% 


4.0% 


10.0% 


High school graduate 


27.3% 


16.9% 


25.7% 


Some college 


10.4% 


4.1% 


7.5% 


Certificate 


3.7% 


1.7% 


3.4% 


Associate degree 


2.5% 


1.8% 


2.2% 


Bachelor’s degree 


4.0% 


4.6% 


5.1% 



This is based on the quintile of the average socioeconomic status of the Census block group in which the 
student’s residence is found. 1 is the highest quintile, and 5 is the lowest. For details, see Crosta, Leinbach, 
Jenkins, Prince, and Whitakker (2006) and WSBCTC (2006). 

^ Vocational and academic intent indicate the type of college program the student means to pursue. If 
vocational, the student intends to pursue workforce training; if academic, the student intends to pursue a 
program that leads to a degree and/or transfer to a four-year institution. Students do not always follow their 
stated intent (see Bailey, Jenkins, & Leinbach, 2006). 
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3.2 Estimates of the probability of earning college credit 

Table 5 shows regression estimates of the differenees in the probability of earning 
eollege eredit (ineluding eollege voeational eredit^) relative to the Non-IB Non- 
Workforce baseline group — those basic skills students who took neither I-BEST nor any 
other workforce course. As shown in Table 5, even after controlling for demographic 
characteristics, enrollment intent and intensity, and previous schooling, I-BEST students’ 
probability of earning college credit was 81 percentage points higher than that of the 
Non-IB Non-Workforce students. There are no significant differences between the 
estimates for I-BEST students who started in ABE/GED and those who started in ESE. 
Both groups appear to have benefited similarly by enrolling in I-BEST. 

Table 5 also shows that Non-IB Workforce students (basic skills students who 
took at least one workforce course but did not participate in I-BEST) also did better than 
Non-IB Non-Workforce students — the former had a probability of earning college credit 
that was 47 percentage points higher than the latter. However, the probability that Non-IB 
Workforce students earned college credit was still not as high as that for I-BEST 
students. Using the regression results and holding the values of all explanatory variables 
other than those corresponding to the three groups of interest at their means, we estimated 
that the probability of earning college credit was 84 percent for I-BEST students, 
compared to 50 percent for Non-IB Workforce students and only 3 percent for Non-IB- 
Non-Workforce students (results not shown in the table). 

Table 6 shows the propensity score matching estimates of the differences in the 
probability of earning college credit between the I-BEST students and both the 
unmatched basic skills population and the matched comparison group. By this PSM 
method, we estimate that the average difference in probability of earning college credit 
between I-BEST students and students in the comparison group was 23 percentage 
points. The mean probability for I-BEST students was 90 percent; it was 67 percent for 
the comparison group (results not shown in the table). 

As mentioned, we cannot statistically compare the results of the regressions with 
those of the PSM analysis because each takes a different approach to selecting 

® Throughout this paper, the findings we report concerning college credits always include college 
vocational credits. 
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appropriate comparison groups. However, the fact that these two different methods yield 
effect size estimates that are similar in magnitude increases our confidence in the results. 
As noted in the appendix, we believe that PSM may give a more accurate estimate of the 
program’s effect on a given outcome. 



Table 5. Logistic Regression Estimates of Differences in 
the Probability of Earning College Credit Relative to 
Non-IB Non-Workforce Students, 2006-08 





Overall 


ABE/GED 


ESL 


I-BEST 


0.81*** 


Q 


0.83*** 


Students 


(0.02) 


(0.03) 


(0.04) 


Non-IB Workforce 


Q /{.y*** 


0.54*** 


0.40** 


Students 


(0.02) 


(0.02) 


(0.04) 


Pseudo R'^2 


0.440 


0.431 


0.427 


Observations 


27,426 


10,058 


17,288 


Note: * p<0.10, ** 


p<0.05, *** p<0.01 






Effect shown for discrete change of variable from 0 to 1 . 




Table 6. PSM Estimates of Differences in the Probability 
of Earning College Credit Relative to Unmatched and 


Matched Non-I-BEST Students, 2006-08 








Average treatment 
effect on the treated 




Unmatched 


(ATT) 


I-BEST 


0.83 


*** 


Q 23*** 


Students 


(0.01) 


(0.03) 



Note: * p<0.10, ** p<0.05, *** p<0.01 
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3.3 Estimates of the number of credits earned 



Table 7 reports the results from estimating differences in the number of college 
and college vocational credits earned by I-BEST and Non-IB Workforce students 
compared to Non-IB Non-Workforce students. Once again using regressions to control 
for the factors indicated in Table 4, the left side of Table 7 shows that I-BEST students 
earned, on average, an estimated 44 more college credits — equivalent to approximately 
one full academic year — than Non-IB Non-Workforce students, and 14 more college 
credits than Non-IB Workforce students. Using the regression results, we estimated that 
I-BEST students earned an average of 45 college credits compared to 3 1 for the Non-IB 
Workforce students and I credit for the Non-IB Non-Workforce group (results not shown 
in table). 

ABE/GED students in I-BEST earned 50 more college credits, and ESE students 
in I-BEST earned 35 more college credits than Non-IB Non-Workforce students. These 
estimates are 19 and 8 credits more than those earned by Non-IB Workforce students who 
were enrolled in ABE/GED and ESE, respectively. 

The three columns on the right side of Table 7 show the results for college 
vocational credits. Even after controlling for demographic characteristics and other 
factors, I-BEST students — overall and by ABE/GED and ESE subgroup — earned more 
vocational credits than Non-IB Non-Workforce students and Non-IB Workforce students. 
I-BEST students earned, on average, an estimated 40 more college vocational credits than 
Non-IB Non- Workforce students and 18 more than Non-IB Workforce students. We 
estimated that, on average, I-BEST students earned 40 vocational credits, Non-IB 
Workforce students earned 22 and Non-IB Non-Workforce students earned less than one. 

ABE/GED I-BEST students earned 45 more college vocational credits than Non- 
IB Non-Workforce students and 21 more than the Non-IB Workforce group. ESE I-BEST 
students earned 31 more college vocational credits than Non-IB Non-Workforce students 
and 14 more than Non-IB Workforce students. 

Table 8 shows the PSM estimate of the difference in the number of credits earned 
by I-BEST students compared with Non-I-BEST students to whom they were matched 
based on similar background characteristics and prior enrollment patterns. The mean 

^ The Washington State community and technical colleges operate on a quarter system. 
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number of credits earned by I-BEST students was 52, compared to an average of 34 for 
the matched comparison group — a difference of 18 credits. I-BEST students earned an 
average of 45 vocational credits, while the matched comparison group earned an average 
of 24 vocational credits, a difference of 21 vocational credits. (Only results on the 
differences in credits earned are shown in the table.) Though not directly comparable, the 
regression and PSM estimates are of similar magnitude, indicating that the results are 
robust. 



Table 7. OLS Regression Estimates of Differences in the Number of Total College and 
College Vocational Credits Earned Relative to Non-IB Non-Workforce Students, 2006-08 



College credits Vocational credits 





Overall 


ABE/GED 


ESL 


Overall 


ABE/GED 


ESL 


I-BEST 


44.35*** 


49.74*** 


34.81*** 


39.64*** 


44.58*** 


30.97*** 


Students 


(1.62) 


(2.23) 


(2.15) 


(1.48) 


(2.03) 


(1.95) 


Non-IB Workforce 


30.36*** 


31.11*** 


26.96*** 


21.52*** 


23.06*** 


17 31*** 


Students 


(1.28) 


(1.57) 


(2.16) 


(1.04) 


(1.32) 


(1.58) 


R-squared 


0.434 


0.455 


0.385 


0.411 


0.427 


0.375 


Observations 


27,426 


10,058 


17,297 


27,426 


10,058 


17,297 



Note: * p<0.10, ** p<0.05, *** p<0.01 



Table 8. PSM Estimates of Differences in the Number of Total College and College 
Vocational Credits Earned Relative to Matched and Unmatched Non-I-BEST Students, 
2006-08 

College credits Vocational credits 

Average treatment Average treatment 

effect on the treated effect on the treated 

Unmatched (ATT) Unmatched (ATT) 



I-BEST 49.60*** 18.48*** 43.18*** 21.39*** 

Students (0.56) (2.71) (0.45) (2.26) 



Note: * p<0.10, ** p<0.05, *** p<0.01 
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3.4 Estimates of the probability of persisting into 2007-08 

We measured persistence into the second academic year, 2007-08, by examining 
whether a student had any transcript record in that year. By this definition, in order to 
have persisted, students must have completed, though not necessarily passed, a course in 
that year. We also considered students as having persisted if they earned an award in 
2006-07, even if they did not persist into 2007-08, because these students experienced a 
successful outcome. The results of the logistic regressions and PSM models for this 
outcome are shown in Tables 9 and 10. 

Using a logistic regression model, we estimated that, on average, I-BEST students 
had a probability of persisting that was 42 percentage points higher than Non-IB Non- 
Workforce students. Among those enrolled in ABE/GED in both these groups, I-BEST 
students had a probability that was 47 percentage points higher. The corresponding 
difference in chances for ESE students was 41 percentage points. A regression that 
compared I-BEST students with Non-IB Workforce students found that the former had a 
probability of persisting that was 13 percentage points higher, with an error of 2 
percentage points. 

Comparing Non-IB Workforce students to Non-IB Non-Workforce students, we 
found that the former had a probability of persisting that was 30 percentage points higher 
the latter. The corresponding probability differences for the ABE/GED and ESE 
subgroups of each of these groups were 35 and 26 percentage points, respectively. 

Using the regression results and holding the value of all variables other than the 
dummy variables corresponding to the three groups of interest at their means, we 
estimated that I-BEST students had an 80 percent probability of persisting into the second 
year (or completing a credential), compared to 68 percent for the Non-IB Workforce 
students and 38 percent for the Non-IB Non-Workforce group (results not shown in the 
table) 

As shown in Table 10, our PSM model of persistence found that I-BEST students 
had a probability of persisting that was 17 percentage points higher than matched 
students. The I-BEST students had a 78 percent probability of persisting, compared to 61 
percent for the matched students (not shown in the table). Here again, the results of the 
PSM model are similar to those of the regressions. 
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Table 9. Logistic Regression Estimates of Differences in 
the Probability of Persisting into 2007-08 Relative to Non- 



IB Non-Workforce Students 




Overall 


ABE/GED 


ESL 


I-BEST 

Students 


Q /j.2*** 
(0.02) 


Q /{.y*** 
(0.02) 


Q 4J*** 

(0.03) 


Non-IB Workforce 
Students 


0.30*** 

(0.02) 


0.35*** 

(0.02) 


0.26*** 

(0.03) 


Pseudo R'^2 


0.058 


0.096 


0.046 


Observations 


27,426 


10,058 


17,297 



Note: * p<0.10, ** p<0.05, *** p<0.01 

Effect shown for discrete change of variable from 0 to 1. 



Table 10. PSM Estimates of Differences in the Probability 
of Persisting into 2007-08 Relative to Matched and 
Unmatched Non-I-BEST Students 

Average Treatment 
Effect on the Treated 
Unmatched (ATT) 



0.39*** 0.17 

(0.02) (0.05) 



Note: * p<0.10, ** p<0.05, *** p<0.01 



I-BEST 

Students 



21 










3.5 Estimates of the probability of earning an award 

Table 1 1 shows the results of the regression estimates of the differences in the 
probability of earning an award by I-BEST and Non-IB Workforce students relative to 
the baseline group, Non-IB Non-Workforce students. Awards may have been earned at 
any time within the two academic years of 2006-07 and 2007-08, and include 
occupational certificates and associate degrees granted by the system. As was shown in 
Table 3, however, virtually all of the awards earned by the students under study here 
were certificates. 

Controlling for student characteristics and prior enrollment patterns, I-BEST 
students had a probability of earning an award that was 5 1 percentage points higher than 
that of Non-IB Non-Workforce students. Non-IB Workforce students had a probability of 
doing so that was 16 percentage points higher than Non-IB Non-Workforce students. 
ABE/GED I-BEST students had a probability of earning an award that was 42 percentage 
points higher than ABE/GED Non-IB Non- Workforce students. Eor the ABE/GED Non- 
IB Workforce group, the corresponding difference is 13 percentage points. Eor I-BEST 
and Non-IB Workforce students enrolled in ESE, the respective differences are 57 
percentage points and 10 percentage points. An additional regression model, not shown in 
the tables, found that I-BEST students had a probability of earning an award that was 35 
percentage points higher than that of Non-IB Workforce students, with an error of 4 
percentage points. 

Based on the regression results, we estimated that I-BEST students had a 5 1 
percent probability of earning an award, compared to 16 percent for the Non-IB 
Workforce students and effectively zero percent for the Non-IB Non-Workforce group 
(results not shown in the table). 

Table 12 shows the PSM model estimate of the increased probability of earning 
an award by I-BEST students compared to matched Non-I-BEST students. Based on this 
model, we found that I-BEST students had a 55 percent probability of earning an award, 
compared to only 15 percent for the matched group (these results are not shown in the 
table) — a 40 percentage point difference. The fact that the PSM estimates are similar to 
those from the regression analysis is reassuring. As mentioned, there are reasons to 
believe that the PSM estimates are more accurate than those of the regression. 
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Table 11. Logistic Regression Estimates of Differences 
in the Probability of Earning an Award Relative to 
Non-IB Non-Workforce Students, 2006-08 





Overall 


ABE/GED 


ESL 


I-BEST 

Students 


0.51*** 

(0.03) 


Q 42*** 

(0.04) 


0.57*** 

(0.06) 


Non-IB Workforce 
Students 


0.16*** 

(0.02) 


0.13*** 

(0.02) 


0.10*** 

(0.03) 


Pseudo R'^2 


0.672 


0.621 


0.759 


Observations 


25,473 


9,541 


14,535 



Note: * p<0.10, ** p<0.05, *** p<0.01 

Effect shown for discrete change of variable from 0 to 1 . 



Table 12. PSM Estimates of Differences in the 
Probability of Earning an Award Relative to Matched 
and Unmatched Non-I-BEST Students, 2006-08 

Average treatment 
effect on the treated 
Unmatched (ATT) 



I-BEST 

Students 



0.55*** 

( 0 . 00 ) 



Q zj.Q*** 

( 0 . 02 ) 










3.6 Estimates of the probability of achieving point gains on basic skills tests 

Table 13 shows the results of estimates of the increased probability that I-BEST 
and Non-IB Workforce students made any point gains in basic skills testing compared to 
Non-IB Non-Workforce students. To make point gains, students needed to show a gain 

Q 

on any of the Comprehensive Adult Student Assessment Systems (CASAS) tests, 
whether in reading, listening, or math. 

The results of logistic regressions indicate that, on average, I-BEST students had a 
probability of making CASAS point gains that was 18 percentage points higher than 
Non-IB Non- Workforce students. Non-IB Workforce students had a likelihood of making 
such gains that was 5 percentage points higher than the Non-IB Non-Workforce group. 
ABE/GED I-BEST students had, on average, a probability that was 21 percentage points 
higher than ABE/GED Non-IB Non- Workforce students. Eor the ABE/GED Non-IB 
Workforce group, the corresponding difference was 9 percentage points. Eor I-BEST and 
Non-IB Workforce students enrolled in ESE, the respective differences were 20 
percentage points and 6 percentage points. Regression analysis (not shown in the table) 
indicates that I-BEST students had a probability that was 13 percentage points higher 
than Non-IB Workforce students, with an error of 2 percentage points on the estimate. 

Based on the regression results, we estimated that the probability of achieving a 
CASAS test score gain was 60 percent for I-BEST students, compared with 47 percent 
for Non-IB Workforce students and 43 percent for Non-IB Non-Workforce students 
(estimates not shown in the tables). 

Table 14 shows the PSM model estimates. The I-BEST students had a 62 percent 
likelihood of achieving a basic skills point gain, compared to a 45 percent probability for 
the matched Non-I-BEST students (these are not shown in the table), a difference of 17 
percentage points. Once again, the similarity between the PSM and regression estimates 
increases the robustness of the findings. 



See http://www.casas.org for more information on these tests. 
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Table 13. Logistic Regression Estimates of Differences in 
the Probability of Achieving Basic Skills Point Gain 
Relative to Non-IB Non-Workforce Students, 2006-08 





Overall 


ABE/GED 


ESL 


I-BEST 

Students 


0.18*** 

(0.02) 


0.21*** 

(0.03) 


0.20*** 

(0.03) 


Non-IB Workforce 
Students 


0.05** 

(0.02) 


Q Q9*** 
(0.02) 


0.06* 

(0.03) 


Pseudo R'^2 


0.047 


0.043 


0.052 


Observations 


27,398 


10,050 


17,297 


Note: * p<0.10, ** p<0.05, *** p<0.01 

Effect shown for discrete change of variable from 0 to 1 . 




Table 14. PSM Estimates of Differences in the Probability 
of Achieving a Basic Skills Point Gain Relative to 
Matched and Unmatched Non-I-BEST Students, 2006-08 




Unmatched 


Average treatment 
effect on the treated 
(ATT) 



I-BEST 

Students 



0.18*** 

( 0 . 02 ) 



Q 27*** 

(0.04) 










4. Conclusion 



Our findings indicate that students participating in I-BEST achieved better 
educational outcomes than did other basic skills students who did not participate in the 
program. I-BEST students were more likely to continue into credit-bearing coursework 
and to earn credits that count toward a college credential. They were more likely to 
persist into the second year, to earn educational awards, and to show point gains in basic 
skills testing. On all outcomes, I-BEST students did moderately or substantially better 
than basic skills students who did not enroll in any workforce course. Moreover, I-BEST 
students had better outcomes than those basic skills students who enrolled in at least one 
workforce course in the same academic year. While the I-BEST group’s comparative 
advantage relative to this latter group was not as large, it was still significant. The 
apparent educational benefits were reaped by I-BEST students who started in either 
ABE/GED or ESE. 

These results are robust with respect to two methodologies: regression (linear or 
logistic) and propensity score matching (PSM). Both methodologies control for observed 
differences between the treated (I-BEST) and comparison groups, but neither can control 
for selection bias that may be due to unobserved differences between the groups. Some of 
these unobserved differences are likely to be related to the selection process, which we 
only partially understand. Thus, while the results show that participation in I-BEST is 
correlated with better educational outcomes over the two-year tracking period, it is 
important to note that they do not provide definitive evidence that the I-BEST program 
caused the superior outcomes. It could be that, because of the way students are selected 
into the program, those who participate have higher motivation or other characteristics 
not measured in this study that make them more likely to succeed. CCRC plans to 
conduct further research to better understand the process by which students are selected 
into I-BEST. We will explore the feasibility of using quasi-experimental methods to 
remedy possible selection bias. The strong positive nature of our findings suggests that an 
experimental test of I-BEST, in which students are randomly assigned to a treatment or a 
control group, might be warranted. 
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As we are able to follow I-BEST students over time and collect more information 
about them, we plan to study their degree attainment and labor force outcomes, such as 
employment rates and earnings. CCRC also plans to extend the study to those students 
who enrolled in an I-BEST program in academic year 2007-08, when the program was 
expanded to include all the institutions in the Washington State community and technical 
college system. We will use data on these students to identify I-BEST programs that have 
superior educational and labor market outcomes, controlling for student characteristics, 
and will conduct field research to identify the practices of effective programs. Einally, we 
will also collect data on program finances to estimate the cost-benefit of the I-BEST 
approach and thus help to assess the feasibility of offering it on a wide scale. 
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Appendix: A Brief Description of Propensity Score Matching 



Propensity score matching (PSM) matches “treated” subjects — in this case, 
students served by I-BEST programs — to selected untreated “control” subjects — in this 
case, basic skills students who did not enroll in I-BEST — who have similar background 
characteristics (Rosenbaum & Rubin, 1983; Winship & Morgan, 1999). PSM conducts 
comparisons between similar pairs of students who differ on whether or not they received 
the treatment, but have similar other observed characteristics. 

PSM first estimates the “propensity score,” which is an assessment of the 
propensity to be treated. It does this by performing a logit or probit regression of the 
treatment dummy variable on all available covariates that, in the researcher’s judgment, 
have the potential to influence the chances of being treated. Treated and untreated 
observations are then matched based on having similar propensity scores, and then the 
average treatment effect on the treated (ATT) can be estimated, which is the average 
difference on an outcome of interest between the matched treated and untreated 
observations.^ The ATT is the average effect of the treatment on the sort of person who 
participates in the program. The effectiveness of PSM is, in part, a function of having 
enough relevant information about the cases to accurately estimate the propensity score, 
and thus accurately estimate the ATT using the matching process that uses this score. 

The matching process selects from those observations for which there is 
“common support,” that is, whose distribution of propensity scores are deemed by the 
algorithm to be sufficiently close to the propensity scores of the treated observations. The 
fact that PSM draws its comparison group from the observations that give common 
support, rather than all observations as is typically done when regression is employed, is 
one reason why PSM estimates may be more accurate. 

In addition, unlike regression, PSM does not assume a particular functional 
relationship between an outcome of interest and the available relevant covariates, 
including treatment status. In contrast, if we estimated a linear regression model of an 
outcome, such as college credits earned, on a treatment status dummy variable (here T 



® There are many variants of PSM, many of which match each treated observation to a weighted set of 
matched untreated observations, rather than a single observation. Herein, we have used probit to estimate 
the propensity score and a local linear regression estimator, which is one method of conducting such a 
match (Todd, 1999). 
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BEST participation) and a number of controls, such as demographics, etc., we would 
obtain an estimate of a fixed effect of treatment across all of the cases (assuming that we 
did not internet the treatment status dummy variable with any other eovariates). PSM 
does not do this; the treatment effect varies with eaeh matehed pair of treated and 
untreated eases, and is the differenee in the outeome between the two oases. 
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